Systems and Methods for Navigation Based on Ordered Sensor Records

ABSTRACT

A method of tracking a medical instrument comprises receiving a model of an anatomical passageway formation and receiving a set of ordered sensor records for the medical instrument. The set of ordered sensor records provide a path history of the medical instrument. The method further comprises registering the medical instrument with the model of the anatomical passageway formation based on the path history.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application61/646,654 filed May 14, 2012, and of U.S. Provisional Application61/759,215 filed Jan. 31, 2013, which are incorporated by referenceherein in their entirety.

FIELD

The present disclosure is directed to systems and methods for navigatinga patient anatomy to conduct a minimally invasive procedure, and moreparticularly to systems and methods for tracking an instrument byregistering temporally ordered sensor information with a model of thepatient anatomy.

BACKGROUND

Minimally invasive medical techniques are intended to reduce the amountof tissue that is damaged during interventional procedures, therebyreducing patient recovery time, discomfort, and deleterious sideeffects. Such minimally invasive techniques may be performed throughnatural orifices in a patient anatomy or through one or more surgicalincisions. Through these natural orifices or incisions clinicians mayinsert interventional instruments (including surgical, diagnostic,therapeutic, or biopsy instruments) to reach a target tissue location.To reach the target tissue location, a minimally invasive interventionalinstrument may navigate natural or surgically created passageways inanatomical systems such as the lungs, the colon, the intestines, thekidneys, the heart, the circulatory system, or the like. To assist theclinician in navigating the instrument through the passageways, modelsof the passageway are prepared using pre-operative or inter-operativeimaging. Current systems for tracking the movement of the instrumentrelative to the modeled passageways are vulnerable to error becauseprecisely locating a portion of the instrument, such as the tip, in adense thicket of narrow anatomical passageways is often challenging.Improved systems and methods are needed for registering a portion of theinstrument to the model of the anatomic passageways to track movement ofthe instrument relative to the model of the anatomy.

SUMMARY

The embodiments of the invention are summarized by the claims thatfollow the description.

In one embodiment, a method of tracking a medical instrument comprisesreceiving a model of an anatomical passageway formation and receiving aset of ordered sensor records for the medical instrument. The set ofordered sensor records provide a path history of the medical instrument.The method further comprises registering the medical instrument with themodel of the anatomical passageway formation based on the path history.

In another embodiment, a system comprises a processor configured forreceiving a model of an anatomical passageway formation and receiving aset of ordered sensor records for a medical instrument equipped with asensor. The set of ordered sensor records provides a path history of themedical instrument. The processor is further configured for registeringthe medical instrument with the model of the anatomical passagewayformation based on the path history.

BRIEF DESCRIPTIONS OF THE DRAWINGS

Aspects of the present disclosure are best understood from the followingdetailed description when read with the accompanying figures. It isemphasized that, in accordance with the standard practice in theindustry, various features are not drawn to scale. In fact, thedimensions of the various features may be arbitrarily increased orreduced for clarity of discussion. In addition, the present disclosuremay repeat reference numerals and/or letters in the various examples.This repetition is for the purpose of simplicity and clarity and doesnot in itself dictate a relationship between the various embodimentsand/or configurations discussed.

FIG. 1 is a robotic interventional system, in accordance withembodiments of the present disclosure.

FIG. 2 illustrates an interventional instrument system utilizing aspectsof the present disclosure.

FIG. 3 a illustrates a path of an interventional instrument within apatient anatomy.

FIG. 3 b illustrates a table of temporally ordered sensor records.

FIG. 4 is a flowchart describing a method of tracking an interventionalinstrument.

FIG. 5 illustrates temporally ordered sensor data and candidate matchpoints along a model of anatomic passageways.

FIG. 6 is a flowchart describing the registration of a sensor trajectoryto an anatomic model.

FIGS. 7-8 illustrate temporally ordered sensor data and candidate matchpoints along a model of anatomic passageways according to embodiments ofthe present disclosure.

FIG. 9 illustrates temporally ordered sensor data registered to themodel of anatomic passageways.

FIG. 10 illustrates a virtual image of the patient anatomy from theviewpoint of the registered interventional instrument.

DETAILED DESCRIPTION

In the following detailed description of the aspects of the invention,numerous specific details are set forth in order to provide a thoroughunderstanding of the disclosed embodiments. However, it will be obviousto one skilled in the art that the embodiments of this disclosure may bepracticed without these specific details. In other instances well knownmethods, procedures, components, and circuits have not been described indetail so as not to unnecessarily obscure aspects of the embodiments ofthe invention. And, to avoid needless descriptive repetition, one ormore components or actions described in accordance with one illustrativeembodiment can be used or omitted as applicable from other illustrativeembodiments.

The embodiments below will describe various instruments and portions ofinstruments in terms of their state in three-dimensional space. As usedherein, the term “position” refers to the location of an object or aportion of an object in a three-dimensional space (e.g., three degreesof translational freedom along Cartesian X, Y, Z coordinates). As usedherein, the term “orientation” refers to the rotational placement of anobject or a portion of an object (three degrees of rotationalfreedom—e.g., roll, pitch, and yaw). As used herein, the term “pose”refers to the position of an object or a portion of an object in atleast one degree of translational freedom and to the orientation of thatobject or portion of the object in at least one degree of rotationalfreedom (up to six total degrees of freedom). As used herein, the term“shape” refers to a set of poses, positions, or orientations measuredalong an object.

Referring to FIG. 1 of the drawings, a robotic interventional system foruse in, for example, surgical, diagnostic, therapeutic, or biopsyprocedures, is generally indicated by the reference numeral 100. Asshown in FIG. 1, the robotic system 100 generally includes aninterventional manipulator assembly 102 for operating an interventionalinstrument 104 in performing various procedures on the patient P. Theassembly 102 is mounted to or near an operating table O. A masterassembly 106 allows the surgeon S to view the surgical site and tocontrol the slave manipulator assembly 102.

The master assembly 106 may be located at a surgeon's console C which isusually located in the same room as operating table O. However, itshould be understood that the surgeon S can be located in a differentroom or a completely different building from the patient P. Masterassembly 106 generally includes an optional support 108 and one or morecontrol device(s) 112 for controlling the manipulator assemblies 102.The control device(s) 112 may include any number of a variety of inputdevices, such as joysticks, trackballs, data gloves, trigger-guns,hand-operated controllers, voice recognition devices, body motion orpresence sensors, or the like. In some embodiments, the controldevice(s) 112 will be provided with the same degrees of freedom as theassociated interventional instruments 104 to provide the surgeon withtelepresence, or the perception that the control device(s) 112 areintegral with the instruments 104 so that the surgeon has a strong senseof directly controlling instruments 104. In other embodiments, thecontrol device(s) 112 may have more or fewer degrees of freedom than theassociated interventional instruments 104 and still provide the surgeonwith telepresence. In some embodiments, the control device(s) 112 aremanual input devices which move with six degrees of freedom, and whichmay also include an actuatable handle for actuating instruments (forexample, for closing grasping jaws, applying an electrical potential toan electrode, delivering a medicinal treatment, or the like).

In alternative embodiments, the robotic system may include more than oneslave manipulator assembly and/or more than one master assembly. Theexact number of manipulator assemblies will depend on the surgicalprocedure and the space constraints within the operating room, amongother factors. The master assemblies may be collocated, or they may bepositioned in separate locations. Multiple master assemblies allow morethan one operator to control one or more slave manipulator assemblies invarious combinations.

An optional visualization system 110 may include an endoscope systemsuch that a concurrent (real-time) image of the surgical site isprovided to surgeon console C. The concurrent image may be, for example,a two- or three-dimensional image captured by an endoscopic probepositioned within the surgical site. In this embodiment, thevisualization system 110 includes endoscopic components that may beintegrally or removably coupled to the interventional instrument 104. Inalternative embodiments, however, a separate endoscope attached to aseparate manipulator assembly may be used to image the surgical site.Alternatively, a separate endoscope assembly may be directly operated bya user, without robotic control. The endoscope assembly may includeactive steering (e.g., via teleoperated steering wires) or passivesteering (e.g., via guide wires or direct user guidance). Thevisualization system 110 may be implemented as hardware, firmware,software, or a combination thereof, which interacts with or is otherwiseexecuted by one or more computer processors, which may include theprocessor(s) of a control system 116.

A display system 111 may display an image of the surgical site andinterventional instruments captured by the visualization system 110. Thedisplay 111 and the master control device(s) 112 may be oriented suchthat the relative positions of the imaging device in the scope assemblyand the interventional instruments are similar to the relative positionsof the surgeon's eyes and hand(s) so the operator can manipulate theinterventional instrument 104 and the master control device(s) 112 as ifviewing the workspace in substantially true presence. True presencemeans that the displayed tissue image appears to an operator as if theoperator was physically present at the imager location and directlyviewing the tissue from the imager's perspective.

Alternatively or additionally, display system 111 may present images ofthe surgical site recorded and/or modeled preoperatively using imagingtechnology such as computerized tomography (CT), magnetic resonanceimaging (MRI), fluoroscopy, thermography, ultrasound, optical coherencetomography (OCT), thermal imaging, impedance imaging, laser imaging,nanotube X-ray imaging, or the like. The presented preoperative imagesmay include two-dimensional, three-dimensional, or four-dimensional(including e.g., time based or velocity based information) images.

In some embodiments, the display system 111 may display a virtualvisualization image in which the actual location of the interventionalinstrument is registered (e.g., dynamically referenced) withpreoperative or concurrent images to present the surgeon S with avirtual image of the internal surgical site at the location of the tipof the surgical instrument.

In other embodiments, the display system 111 may display a virtualvisualization image in which the actual location of the interventionalinstrument is registered with prior images (including preoperativelyrecorded images) or concurrent images to present the surgeon S with avirtual image of an interventional instrument at the surgical site. Animage of a portion of the interventional instrument may be superimposedon the virtual image to assist the surgeon controlling theinterventional instrument.

As shown in FIG. 1, a control system 116 includes at least one processor(not shown), and typically a plurality of processors, for effectingcontrol between the slave surgical manipulator assembly 102, the masterassembly 106, the visualization system 110, and the display system 111.The control system 116 also includes programmed instructions (e.g., acomputer-readable medium storing the instructions) to implement some orall of the methods described herein. While control system 116 is shownas a single block in the simplified schematic of FIG. 1, the system maycomprise a number of data processing circuits (e.g., on the slavesurgical manipulator assembly 102 and/or on the master assembly 106),with at least a portion of the processing optionally being performedadjacent the slave surgical manipulator assembly, a portion beingperformed the master assembly, and the like. Any of a wide variety ofcentralized or distributed data processing architectures may beemployed. Similarly, the programmed instructions may be implemented as anumber of separate programs or subroutines, or they may be integratedinto a number of other aspects of the robotic systems described herein.

In one embodiment, control system 116 supports wireless communicationprotocols such as Bluetooth, IrDA, HomeRF, IEEE 802.11, DECT, andWireless Telemetry.

In some embodiments, control system 116 may include one or more servocontrollers to provide force and torque feedback from the interventionalinstruments 104 to one or more corresponding servomotors for the controldevice(s) 112. The servo controller(s) may also transmit signalsinstructing manipulator assembly 102 to move instruments which extendinto an internal surgical site within the patient body via openings inthe body. Any suitable conventional or specialized servo controller maybe used. A servo controller may be separate from, or integrated with,manipulator assembly 102. In some embodiments, the servo controller andmanipulator assembly are provided as part of a robotic arm cartpositioned adjacent to the patient's body.

Each manipulator assembly 102 supports a interventional instrument 104and may comprise a kinematic structure of one or more non-servocontrolled links (e.g., one or more links that may be manuallypositioned and locked in place, generally referred to as a set-upstructure) and a robotic manipulator. The robotic manipulator assembly102 is driven by a plurality of actuators (e.g., motors). These motorsactively move the robotic manipulators in response to commands from thecontrol system 116. The motors are further coupled to the interventionalinstrument so as to advance the interventional instrument into anaturally or surgically created anatomical orifice and to move thedistal end of the interventional instrument in multiple degrees offreedom, which may include three degrees of linear motion (e.g., linearmotion along the X, Y, Z Cartesian axes) and three degrees of rotationalmotion (e.g., rotation about the X, Y, Z Cartesian axes). Additionally,the motors can be used to actuate an articulable end effector of theinstrument for grasping tissue in the jaws of a biopsy device or thelike.

FIG. 2 illustrates a minimally invasive system 200 utilizing aspects ofthe present disclosure. The system 200 may be incorporated into arobotic interventional system, such as system 100. Alternatively, thesystem 200 may be used for non-robotic exploratory procedures or inprocedures involving traditional manually operated interventionalinstruments, such as endoscopy. The system 200 includes a cathetersystem 202 (e.g., part of the instrument 104) coupled by an interfaceunit 204 to a tracking system 206. A navigation system 210 (e.g., partof the control system 116) processes information from a virtualvisualization system 208 and the tracking system 206 to generate avirtual image display on a display system 214 (e.g., part of the displaysystem 111). The system 200 may further include optional operation andsupport systems (not shown) such as imaging systems, illuminationsystems, steering control systems, irrigation systems, and/or suctionsystems.

The catheter system 202 includes an elongated flexible body 216 having aproximal end 217 and a distal end 218. In one embodiment, the flexiblebody 216 has an approximately 3 mm outer diameter. Other flexible bodyouter diameters may be larger or smaller. The catheter system 202includes a sensor system which includes a position sensor system 220(e.g., an electromagnetic (EM) sensor system) and/or a shape sensorsystem 222 for determining the position, orientation, speed, pose,and/or shape of the catheter tip at distal end 218 and/or of one or moresegments 224 along the body 216. The entire length of the body 216,between the distal end 218 and the proximal end 217 may be effectivelydivided into the segments 224. The position sensor system 220 and theshape sensor system 222 interface with the tracking system 206. Thetracking system 206 may be implemented as hardware, firmware, softwareor a combination thereof which interact with or are otherwise executedby one or more computer processors, which may include the processors ofa control system 116.

The position sensor system 220 may be an EM sensor system that includesone or more conductive coils that may be subjected to an externallygenerated electromagnetic field. Each coil of the EM sensor system 220then produces an induced electrical signal having characteristics thatdepend on the position and orientation of the coil relative to theexternally generated electromagnetic field. In one embodiment, the EMsensor system may be configured and positioned to measure six degrees offreedom, e.g., three position coordinates X, Y, Z and three orientationangles indicating pitch, yaw, and roll of a base point or five degreesof freedom, e.g., three position coordinates X, Y, Z and two orientationangles indicating pitch and yaw of a base point. Further description ofan EM sensor system is provided in U.S. Pat. No. 6,380,732, filed Aug.11, 1999, disclosing “Six-Degree of Freedom Tracking System Having aPassive Transponder on the Object Being Tracked,” which is incorporatedby reference herein in its entirety.

The shape sensor system 222 includes an optical fiber aligned with theflexible body 216 (e.g., provided within an interior channel (not shown)or mounted externally). The tracking system 206 may be coupled to aproximal end of the optical fiber. In one embodiment, the optical fiberhas a diameter of approximately 200 μm. In other embodiments, thedimensions may be larger or smaller.

The optical fiber of the shape sensor system 222 forms a fiber opticbend sensor for determining the shape of the catheter system 202. In onealternative, optical fibers including Fiber Bragg Gratings (FBGs) areused to provide strain measurements in structures in one or moredimensions. Various systems and methods for monitoring the shape andrelative position of an optical fiber in three dimensions are describedin U.S. patent application Ser. No. 11/180,389, filed Jul. 13, 2005,disclosing “Fiber optic position and shape sensing device and methodrelating thereto;” U.S. Provisional Pat. App. No. 60/588,336, filed onJul. 16, 2004, disclosing “Fiber-optic shape and relative positionsensing;” and U.S. Pat. No. 6,389,187, filed on Jun. 17, 1998,disclosing “Optical Fibre Bend Sensor,” which are incorporated byreference herein in their entireties. In other alternatives, sensorsemploying other strain sensing techniques such as Rayleigh scattering,Raman scattering, Brillouin scattering, and Fluorescence scattering maybe suitable. In other alternative embodiments, the shape of the cathetermay be determined using other techniques. For example, if the history ofthe catheter's distal tip pose is stored for an interval of time that issmaller than the period for refreshing the navigation display or foralternating motion (e.g., inhalation and exhalation), the pose historycan be used to reconstruct the shape of the device over the interval oftime. As another example, historical pose, position, or orientation datamay be stored for a known point of an instrument along a cycle ofalternating motion, such as breathing. This stored data may be used todevelop shape information about the catheter. Alternatively, a series ofpositional sensors, such as EM sensors, positioned along the cathetercan be used for shape sensing. Alternatively, a history of data from apositional sensor, such as an EM sensor, on the instrument during aprocedure may be used to represent the shape of the instrument,particularly if an anatomical passageway is generally static.Alternatively, a wireless device with position or orientation controlledby an external magnetic field may be used for shape sensing. The historyof its position may be used to determine a shape for the navigatedpassageways.

In this embodiment, the optical fiber may include multiple cores withina single cladding. Each core may be single-mode with sufficient distanceand cladding separating the cores such that the light in each core doesnot interact significantly with the light carried in other cores. Inother embodiments, the number of cores may vary or each core may becontained in a separate optical fiber.

In some embodiments, an array of FBG's is provided within each core.Each FBG comprises a series of modulations of the core's refractiveindex so as to generate a spatial periodicity in the refraction index.The spacing may be chosen so that the partial reflections from eachindex change add coherently for a narrow band of wavelengths, andtherefore reflect only this narrow band of wavelengths while passingthrough a much broader band. During fabrication of the FBG's, themodulations are spaced by a known distance, thereby causing reflectionof a known band of wavelengths. However, when a strain is induced on thefiber core, the spacing of the modulations will change, depending on theamount of strain in the core. Alternatively, backscatter or otheroptical phenomena that vary with bending of the optical fiber can beused to determine strain within each core.

Thus, to measure strain, light is sent down the fiber, andcharacteristics of the returning light are measured. For example, FBG'sproduce a reflected wavelength that is a function of the strain on thefiber and its temperature. This FBG technology is commercially availablefrom a variety of sources, such as Smart Fibres Ltd. of Bracknell,England. Use of FBG technology in position sensors for robotic surgeryis described in U.S. Pat. No. 7,930,065, filed Jul. 20, 2006, disclosing“Robotic Surgery System Including Position Sensors Using Fiber BraggGratings,” which is incorporated by reference herein in its entirety.

When applied to a multicore fiber, bending of the optical fiber inducesstrain on the cores that can be measured by monitoring the wavelengthshifts in each core. By having two or more cores disposed off-axis inthe fiber, bending of the fiber induces different strains on each of thecores. These strains are a function of the local degree of bending ofthe fiber. For example, regions of the cores containing FBG's, iflocated at points where the fiber is bent, can thereby be used todetermine the amount of bending at those points. These data, combinedwith the known spacings of the FBG regions, can be used to reconstructthe shape of the fiber. Such a system has been described by LunaInnovations. Inc. of Blacksburg, Va.

As described, the optical fiber may be used to monitor the shape of atleast a portion of the catheter system 202. More specifically, lightpassing through the optical fiber is processed by the tracking system206 for detecting the shape of the catheter system 202 and for utilizingthat information to assist in surgical procedures. The tracking system206 may include a detection system for generating and detecting thelight used for determining the shape of the catheter system 202. Thisinformation, in turn, in can be used to determine other relatedvariables, such as velocity and acceleration of the parts of aninterventional instrument. The sensing may be limited only to thedegrees of freedom that are actuated by the robotic system, or may beapplied to both passive (e.g., unactuated bending of the rigid membersbetween joints) and active (e.g., actuated movement of the instrument)degrees of freedom.

The flexible body 216 may optionally house an image capture probe 226.The image capture probe 226 includes a tip portion with a stereoscopicor monoscopic camera disposed near the distal end 218 of the flexiblebody 216 for capturing images (including video images) that aretransmitted to and processed by the navigation system 210 for display.The image capture probe 226 may include a cable coupled to the camerafor transmitting the captured image data. Alternatively, the imagecapture instrument may be a fiber-optic bundle, such as a fiberscope,that couples to the imaging system. The image capture instrument may besingle or multi-spectral, for example capturing image data in thevisible spectrum, or capturing image data in the visible and infrared orultraviolet spectrums.

The body 216 may also house cables, linkages, or other steering controls(not shown) that extend between the interface 204 and the tip distal end218 to controllably bend or turn the distal end 218 as shown for exampleby the dotted line versions of the distal end. The catheter system maybe steerable or, alternatively, may be non-steerable with no integratedmechanism for operator control of the instrument bending. The flexiblebody 216 may further house control mechanisms (not shown) for operatinga surgical end effector or another working distal part that ismanipulable for a medical function, e.g., for effecting a predeterminedtreatment of a target tissue. For instance, some end effectors have asingle working member such as a scalpel, a blade, an optical fiber, oran electrode. Other end effectors may include pair or plurality ofworking members such as forceps, graspers, scissors, or clip appliers,for example. Examples of electrically activated end effectors includeelectrosurgical electrodes, transducers, sensors, and the like. Also oralternatively, the flexible body 216 can define one or more lumensthrough which interventional instruments can be deployed and used at atarget surgical location.

The virtual visualization system 208 provides navigation assistance tothe catheter system 202. Virtual navigation using the virtualvisualization system is based upon reference to an acquired datasetassociated with the three dimensional structure of the anatomicalpassageways. More specifically, the virtual visualization system 208processes images of the surgical site recorded and/or modeled usingimaging technology such as computerized tomography (CT), magneticresonance imaging (MRI), fluoroscopy, thermography, ultrasound, opticalcoherence tomography (OCT), thermal imaging, impedance imaging, laserimaging, nanotube X-ray imaging, or the like. Software is used toconvert the recorded images into a two dimensional or three dimensionalmodel of a partial or an entire anatomical organ or anatomical region.The model describes the various locations and shapes of the passagewaysand their connectivity. The images used to generate the model may berecorded preoperatively or intra-operatively during a clinicalprocedure. In an alternative embodiment, a virtual visualization systemmay use standard models (i.e., not patient specific) or hybrids of astandard model and patient specific data. The model and any virtualimages generated by the model may represent the static posture of adeformable anatomic region during one or more phases of motion (e.g.,during an inspiration/expiration cycle of a lung).

During a virtual navigation procedure, the sensor systems may be used tocompute an approximate location of the instrument with respect to thepatient anatomy. The location can be used to produce both macro-leveltracking images of the patient anatomy and virtual internal images ofthe patient anatomy. Various systems for using fiber optic sensors toregister and display an interventional implement together withpreoperatively recorded surgical images, such as those from a virtualvisualization system, are known. For example U.S. patent applicationSer. No. 13/107,562, filed May 13, 2011, disclosing, “Medical SystemProviding Dynamic Registration of a Model of an Anatomical Structure forImage-Guided Surgery,” which is incorporated by reference herein in itsentirety, discloses one such system.

Existing systems and methods for locating the tip of the catheter andaccurately registering it with an anatomic passageway model from thevirtual visualization system are often not entirely adequate,particularly in portions of the anatomy, such as the lungs, where airwaypassages are narrow and densely arranged. These prior approaches aresubject to inaccuracies due to measurement precision limits and anatomicmovement (e.g., patient breathing or tissue deformation in response tosurgical instrumentation). For example, the positional informationprovided by an EM sensor may be accurate to approximately +/−3 mm withrespect to a lung after registration to patient anatomy model, usingreference fiducials on the chest. When motion of the lungs isconsidered, the error may increase up to approximately +/−10 mm or more.These errors may cause the instrument tip to become registered to thewrong passageway of the anatomic model or may result in the sensorsystem generating a location for the tip of the catheter that is outsideof an anatomic passageway. As the catheter moves through the passagewaysystem, these errors can cause the instrument to unrealistically jumpbetween modeled passageways as the catheter tip is advanced. If thelocation of the tip of the catheter is incorrectly determined, thevirtual visualization system will return an incorrect virtual view fromthe distal end of the catheter (e.g., showing the probe in the wrongbranch of the airway tree) therefore giving inaccurate guidance to theclinician.

In embodiments of the present disclosure, temporal or spatial orderingof the sensor data is utilized in a probabilistically reasoningframework to improve the limitations mentioned above. Temporally orderedhistorical data (i.e, chronologically sequenced data records) from thesensor system may be processed by the navigation system 210 to registerand track the catheter tip with respect to an anatomic passageway modelfrom the virtual visualization system 208. The temporally orderedhistorical data may be generated, for example, from an EM sensor locatedon the catheter. Alternatively or additionally, spatially ordered data(i.e., data records arranged in order along the shape of catheter andoptionally recorded at fixed intervals) from the sensor system may beprocessed by the navigation system to register and track the catheterwith respect to an anatomic passageway model. The spatially ordered datamay be generated, for example, from a fiber shape sensor extendingwithin the catheter. A temporally ordered data set, a spatially ordereddata set, or a single set of shape data can provide a path history ofthe catheter. Other information, such as airway topology and physicalconstraints, may also be used with the ordered sensor data to provide aprobabilistic method for tracking the catheter tip through the anatomicpassageways. The use of this additional information may be used withcurrent sensor information to more accurately locate the catheter tiprelative to the model and generate a virtual image of the anatomy thatcorresponds to the true location of the catheter tip. The cliniciannavigating the instrument is, thus, less likely to guide the catheter tounintended locations, minimizing the need for time-consuming andpotentially injurious backtracking.

FIG. 3 a illustrates a path 300 of the tip of an interventionalinstrument (such as catheter system 202) within branches 302, 304, 306of a patient anatomy. As the instrument is advanced within the patientanatomy, sensor records are generated at time instances T1-T6. At T1,the tip is within the branch 302. At T2, the tip has entered the branch306. At T3, the tip is within the branch 306. At T4, the tip hasreversed directions and is being withdrawn from the branch 306. At T5,the tip has entered the branch 304. At T6, the tip is within the branch304.

FIG. 3 b illustrates a table 320 of temporally ordered sensor records322-332 associated with time instances T1-T6, respectively, along thetrajectory of path 300. In this embodiment, for example, the sensorrecord 322 is gathered at time T1 and catalogs the position x1, the unitorientation vector u1, and shape data h1 of a portion of the instrumentsuch as the tip. The position and unit orientation vector may be basedupon a position sensor, such as an sensor, and the shape data may bebased upon a shape sensor. The position data may include, for example,three degrees of freedom information (e.g., X, Y, Z coordinates in aCartesian system). The unit orientation information may include threedegrees of freedom information (e.g., pitch, yaw, roll).

The common table format of FIG. 3 b, is for illustration only and posesno limitations on how the data associated with each time instance isstored. For example, each type of data (e.g., position, orientation,shape) may be maintained in separate databases. Alternatively, dataassociated with different sensor systems (e.g., position sensor system,shape sensor system) may be maintained separately and for different timeinstances. Also, no limitation is imposed on the type of data that maybe stored for each time instance. Sensor records may be stored asindividual files or may be stored in one or more databases.Alternatively, some other data management system may be used. Thepresent disclosure imposes no limitations on how the sensor records arestored or what information is associated with a sensor record.

FIG. 4 is a flowchart describing a method of tracking an interventionalinstrument (such as catheter system 202) with respect to a model of apatient anatomy. At 402, a full or partial set of sensor records arereceived in a temporal order at a processor (e.g. the processors ofcontrol system 116). Alternatively, the sensor records may include timesequencing information that allows the records to be arranged in atemporal order after receipt.

At 404, the sensor records are, optionally, filtered to remove sensorrecords from the set or to remove information from selected sensorrecords in the set. For example, the sensor record set may be edited toinclude only those sensor records along a direct path from a root record(e.g., T1) to a current record (e.g., T6). In this example, the sensorrecords for T2, T3, and T4 would be dropped, with the sensor record forT1, T5, and T6 remaining in the set to be analyzed. Additionally oralternatively, the sensor record set may be edited to include only dataassociated with a selected phase of cyclical patient movement (e.g.,breathing motion). For example, a gating signal associated with a phaseof respiration, such as expiration, may be provided that allows onlysensor records associated with the lung in a state of expiration to beincluded in a subset of data for analysis. Gating is used to minimizethe deviation of the sensor data from the model and eliminate outlyingdata records, providing a more coherent data set associated with theeffective trajectory of the interventional instrument. Furtherdescription of systems and methods for gathering and analyzing dataassociated with cyclical patient movements is provided in U.S. patentapplication Ser. No. 13/297,066, filed Nov. 15, 2011, disclosing “Methodand System for Determining Information of Extrema During Expansion andContraction Cycles of an Object,” which is incorporated by referenceherein in its entirety. The filtered subset of sensor records may beused to determine a velocity for the instrument tip between timeinstances. The calculated velocity vector may or may not be the same asthe pointing direction associated with the sensor record. In some cases,the velocity vector may match the path direction better than thepointing direction associated with the sensor record. Estimatingvelocity from incoming positional data (e.g., using a Kalman filter) hasinherent latency. Additionally or alternatively velocity estimation canbe computed by positional data both before and after the time instantbeing estimated (e.g., using Kalman smoothing).

To limit the influence of long pauses in motion or very slow velocities,at 406, the sensor records may be further subsampled at a selectedspatial resolution. For example, a spatial resolution of one sample permillimeter may be selected for subsampling the sensor records. Thespatial resolution may be selected to correspond the spatial resolutionof the imaging technology used to generate the model (e.g., the CTspatial resolution). Spatial subsampling would eliminate a repeat sensorrecord from the subset if the tip of the instrument is stopped for anextended period of time. Trajectory orientation estimation may,optionally, be performed after spatial subsampling to avoid an undefinedvelocity when the device does not move or moves very slowly. Aftersubsampling at 406, the resultant subset of sensor records may have anequal spatial spacing.

At 408, the model of the anatomic passageways is received from thevirtual visualization system. At 410, the trajectory of the tip of thecatheter is registered with the model of the anatomic passageways.Generally, for each passageway in the model, a centerline or any othersampled line through the modeled passageway is determined from themodel. The sensor trajectory is registered with the model by matchingthe subset of sensor records to the sampled line through the modeledpassageway.

In one illustrative example, the subset of sensor records are temporallyordered and spatially sampled EM sensor records generated by an EMsensor located near the tip of an interventional instrument. In thisembodiment, centerlines through the modeled anatomic passageways arecomprised of a series of centerline points labeled Li, where i=1, . . ., M and M is the total number of centerline points. Each centerlinepoint Li has a three-dimensional coordinate, Pi, and a unit orientationvector Oi. FIG. 5 illustrates centerlines 500, 502, 504 formed ofcenterline points Li. Each EM sensor record includes an observationlabeled Xi=[xi, ui] where xi is the three-dimensional coordinate for theobservation and ui is the unit orientation vector for the observation.To determine the centerline point Li that is best matched to each EMobservation, a solution S=[s1, . . . , sn] is calculated. The solutionis expressed as S*=argmax(P(XX|S)), where

${P\left( {{XX}S} \right)} = {\prod\limits_{i = 1}^{n}\; {{P\left( {{Xi}{si}} \right)}{\prod\limits_{i = 2}^{n}\; {{P\left( {s_{i}s_{i\; 1}} \right)}{{P\left( s_{1} \right)}.}}}}}$

As shown in FIG. 6, a process 600 for registering the sensor trajectorywith the model includes at 602, generating a hypothesis by identifyingcandidate centerline points that correspond to, for example thetemporally ordered EM sensor records. At 604, a candidate pointlikelihood computation is performed based upon candidate point positionand orientation. At 606, a transition probability between candidatepoints is calculated. At 608, the state likelihood computations andtransition probability computations are used to optimize the modelregistration.

In one embodiment, we constrain the candidates to be on the centerlineof the passage. One can also use other methods to generate discretecandidates. In greater detail, the initial step of hypothesis generationincludes best matching each sensor observation Xi to a centerline pointLi. As shown in FIG. 5, for each Xi (X1, X2, X3, . . . ), there is afinite set of candidate centerline points Ki (K1, K2, K3, . . . ) (i.e.,subset of the centerline points Li) within each sensor error boundary Bi(B1, B2, B3, . . . ). The candidate points Ki for each boundary set mayoverlap. Each centerline point Ki is a hypothetical possible match forthe sensor observation Xi. Although FIG. 5 describes multiple hypothesesfor each observation Xi, in other various embodiments, a single bestmatch centerline point K for each piece of the passage may be chosenwithin each error boundary B.

Next, a likelihood computation is performed by matching position andorientation information for each EM sensor observation Xi to positionand orientation information for a centerline point Li. Likelihoodmeasures how likely an EM observation is generated by the state (i.e.,how likely one of the centerline points in the set Ki is a match for theEM sensor observation Xi). The likelihood computation includes theposition matching term (xi) and the orientation matching term (ui) forsensor record Xi:

${P\left( {X_{i}s_{i}} \right)} \propto {\exp \left\{ {- \frac{{{x_{i} - P_{si}}}\hat{}2}{\sigma_{1}^{2}}} \right\} \exp {\left\{ {- \frac{\left( {\arccos \; {{dot}\left( {u_{i,}\sigma_{si}} \right)}} \right)\hat{}2}{\sigma_{2}^{2}}} \right\}.}}$

Next, in the process of best matching each temporally ordered EM sensorobservation Xi to a centerline point Li is a transition probabilitycomputation. Transition probability measures the likelihood of one statechange to the next state. For example, transition probability maymeasure the likelihood that a sequential pair of candidate centerlinepoints is best matched to a pair of EM sensor observations.Alternatively, the transition probability may be defined without sensorobservations. Transition probability computation may apply penalties tounlikely sequential candidate centerline points or otherwise placeconstraints on sequential pairs. For example, a jump between a candidatecenterline points that are far away from each other would be penalizedas compared to candidate centerline points that are closer together.Other constraints may also be encoded. For example, a penalty may beplaced on state transitions that do not consider sensor inputs.

${P\left( {s_{i}s_{i - 1}} \right)} = {\exp {\left\{ {- \frac{{{P_{si} - P_{{si} - 1}}}^{2}}{2\; \sigma_{3}^{2}}} \right\}.}}$

As another example, a penalty may be placed on state transitionsconsidering sensor input by matching the vector of sensor input and thevector of the states.

${P\left( {s_{i}s_{i - 1}} \right)} = {\exp \left\{ {- \frac{{{\left( {P_{si} - P_{{si} - 1}} \right) - \left( {x_{i} - x_{i - 1}} \right)}}^{2}}{2\; \sigma_{3}^{2}}} \right\}}$

As another example, state transitions may be confined to occur only onthe centerline tree structure, where D( ) is the distance along the treestructure.

${P\left( {s_{i}s_{i - 1}} \right)} = {\exp \left\{ {- \frac{\left( {{{x_{i} - x_{i - 1}}} - {D\left( {s_{i},s_{i - 1}} \right)}} \right)^{2}}{2\; \sigma_{3}^{2}}} \right\}}$

This enforces the topological constraint on the matching methodology.The direction of the motion may also be used to further refine thematching.

When the sensor observation is associated with a significant amount ofnoise, the distance ∥x_(i)−x_(i-1)∥ is always greater than the actualD(s_(i), s_(i-1)). This may lead to a tendency to choose sequentialcandidate centerline points with a larger than actual distance in orderto match the noisy sensor input. To minimize the effect of this noise,the sensor motion vector may be projected to the centerline direction.For example, in the following expression, the second term penalizes ahigh angle difference between the sensor motion vector and the statemotion vector (i.e., the candidate centerline point motion vector).

${P\left( {s_{i}s_{i - 1}} \right)} = {\exp \left\{ {- \frac{\left( {{{x_{i} - x_{i - 1}}} - {D\left( {s_{i},s_{i - 1}} \right)}} \right)^{2}}{2\; \sigma_{3}^{2}}} \right\} \exp \left\{ {- \frac{{{\left( {x_{i}\overset{\sim}{- x_{i - 1}}} \right) - \left( {P_{si}\overset{\sim}{- P_{{si} - 1}}} \right)}}\hat{}2}{2\; \sigma_{4}^{2}}} \right\}}$

The direction of the path within the centerline tree structure may alsobe used to augment the matching process. For example, if the directionsassociated with the candidate centerline points change but the actualmotor or manual manipulation of the instrument never changed direction,then the pair of candidate centerline points should be consideredimpossible. The aforementioned pair-wise constraints may enforce sometransition smoothing, but additional optimization may further refine thematching.

Transition probability may also consider the consistency of error. Thesensor observation records are not truly independent due to the smoothnature of deformation. Gross deformation may be compensated in theprobability calculation by using term below in the transitionprobability. It is in favor of slow changing of the error vectors(vector from state to sensor input)

$\exp {\left\{ {- \frac{{{\left( {x_{i} - P_{si}} \right) - \left( {x_{i - 1} - P_{{si} - 1}} \right)}}^{2}}{2\; \sigma_{5}^{2}}} \right\}.}$

Global optimization may be solved, for example, by entering the computedlikelihoods and transition probabilities into a table and using dynamicprogramming (e.g., the Viterbi algorithm) to compute a solution. Foreach time instant T, the best trajectory estimate is based upon theprior temporally ordered sensor records.

FIG. 7 illustrates temporally ordered sensor data 700 and candidatematch points 702 along a series of centerline points 704 that correspondto anatomic passageways in an anatomic model. In this illustration, thetemporally ordered sensor data 700 and the candidate points 702 arematched only based upon their individual position without pair-wiseconstraints, which generates error in that candidate points are locatedin discontinuous branches of the anatomic passageway tree. FIG. 8illustrates a more optimized registration using the techniques describedabove. In this embodiment, the temporally ordered sensor data 800 ismatched to candidate points 802 on contiguous branches of the anatomicpassageway tree.

FIG. 9 graphically illustrates the registration of the temporallyordered sensor data 900 and the candidate match points 902. The mostcurrent sensor data at point 904 indicates that the tip of the catheteris currently located just proximal of a bifurcation in the modeledpassageways. As shown in FIG. 10, with the tip of the catheterregistered to the model, a virtual view 950 of the anatomic passagewaybifurcation from perspective of the tip of the catheter is generated fordisplay to the clinician. There may be other options to generate avirtual view indicative of the tip pose with respect to the model. Asthe catheter is advanced through the anatomic passageways, the processesdescribed above are rapidly repeated to provide the clinician with avirtual view of the anatomic passageways that corresponds to the currentpose of the tip of the catheter. The virtual views allow the clinicianto guide the catheter through the passageways without reliance on realcameras. For example, when the passageways become too small toaccommodate a camera or when the camera view becomes obstructed by bloodor mucous, the catheter can be guided by the virtual visualizationsystem.

Although the registration systems and methods have been described hereinwith respect to tele-operated or hand operated interventional systems,these registration systems and methods will find application in avariety of medical and non-medical instruments in which accurateinstrument image registration is required.

Although the systems and methods of this disclosure have been describedfor use in the connected bronchial passageways of the lung, they arealso suited for navigation and treatment of other tissues, via naturalor surgically created connected passageways, in any of a variety ofanatomical systems including the colon, the intestines, the kidneys, thebrain, the heart, the circulatory system, or the like. The methods andembodiments of this disclosure are also suitable for non-interventionalapplications.

One or more elements in embodiments of the invention may be implementedin software to execute on a processor of a computer system such ascontrol system 116. When implemented in software, the elements of theembodiments of the invention are essentially the code segments toperform the necessary tasks. The program or code segments can be storedin a processor readable storage medium or device that may have beendownloaded by way of a computer data signal embodied in a carrier waveover a transmission medium or a communication link. The processorreadable storage device may include any medium that can storeinformation including an optical medium, semiconductor medium, andmagnetic medium. Processor readable storage device examples include anelectronic circuit; a semiconductor device, a semiconductor memorydevice, a read only memory (ROM), a flash memory, an erasableprogrammable read only memory (EPROM); a floppy diskette, a CD-ROM, anoptical disk, a hard disk, or other storage device, The code segmentsmay be downloaded via computer networks such as the Internet, Intranet,etc.

Note that the processes and displays presented may not inherently berelated to any particular computer or other apparatus. The requiredstructure for a variety of these systems will appear as elements in theclaims. In addition, the embodiments of the invention are not describedwith reference to any particular programming language. It will beappreciated that a variety of programming languages may be used toimplement the teachings of the invention as described herein.

While certain exemplary embodiments of the invention have been describedand shown in the accompanying drawings, it is to be understood that suchembodiments are merely illustrative of and not restrictive on the broadinvention, and that the embodiments of the invention not be limited tothe specific constructions and arrangements shown and described, sincevarious other modifications may occur to those ordinarily skilled in theart.

What is claimed is:
 1. A method of tracking a medical instrument, themethod comprising: receiving a model of an anatomical passagewayformation; receiving a set of ordered sensor records for the medicalinstrument, the set providing a path history of the medical instrument;and registering the medical instrument with the model of the anatomicalpassageway formation based on the path history.
 2. The method of claim 1further comprising determining a relationship between a first sensorrecord of the set of ordered sensor records, a second sensor record ofthe set of ordered sensor records, a first candidate match point definedwithin the model of the anatomical passageway, and a second candidatematch point defined within the model of the anatomical passageway. 3.The method of claim 2 wherein determining the relationship includesdetermining a transition probability between the first and secondcandidate match points.
 4. The method of claim 1 further comprisingtracking the movement of the medical instrument relative to the model ofthe anatomical passageway formation.
 5. The method of claim 1 whereinthe set of ordered sensor records include a set of temporally orderedsensor records.
 6. The method of claim 1 wherein the set of orderedsensor records include a set of spatially ordered sensor records.
 7. Themethod of claim 1 wherein receiving the model of the anatomicalpassageway formation includes receiving the model formed from a set ofthree-dimensional volumetric images.
 8. The method of claim 1 whereinreceiving a set of ordered sensor records includes receiving the setfrom an electromagnetic sensor coupled to the medical instrument.
 9. Themethod of claim 1 wherein the set of ordered sensor records includestemporally ordered pose observations for a tip of the medicalinstrument.
 10. The method of claim 1 wherein the set of ordered sensorrecords includes temporally ordered shape observations for a portion ofthe medical instrument.
 11. The method of claim 1 wherein registeringincludes filtering the received set of ordered sensor records.
 12. Themethod of claim 11 wherein filtering the received set of ordered sensorrecords includes creating a subset of the ordered sensor records thatincludes only sensor data recorded at a selected phase of anatomicalmovement.
 13. The method of claim 12 wherein the selected phase ofanatomical movement is an expiration phase of a breathing cycle of alung.
 14. The method of claim 3 wherein determining the transitionprobability includes determining a distance between the first and secondcandidate match points.
 15. The method of claim 3 wherein determiningthe transition probability includes determining a distance between afirst sensor record associated with the first candidate match point anda second sensor record associated with the second candidate match point.16. The method of claim 3 wherein determining the transition probabilityincludes determining whether the first and second candidate match pointsare in contiguous passageways of the model of the anatomical passagewayformation.
 17. The method of claim 1 further comprising spatiallysubsampling the set of ordered sensor records.
 18. The method of claim 1wherein registering includes comparing position data from at least oneof the sensor records in the set of ordered sensor records to at leastone of the candidate match points defined within a passageway in themodel of the anatomical passageway formation.
 19. The method of claim 1wherein registering includes comparing orientation vector data from atleast one of the sensor records in the set of ordered sensor records toat least one of the candidate match points defined within a passagewayin the model of the anatomical passageway formation.
 20. The method ofclaim 1 further comprising displaying an internal virtual view of theanatomical passageway formation.
 21. A system comprising: a processorconfigured for receiving a model of an anatomical passageway formation;receiving a set of ordered sensor records for a medical instrumentequipped with a sensor, the set providing a path history of the medicalinstrument; and registering the medical instrument with the model of theanatomical passageway formation based on the path history.
 22. Thesystem of claim 21 further comprising the medical instrument equippedwith the sensor.
 23. The system of claim 21 wherein the medicalinstrument includes an elongated flexible body.
 24. The system of claim21 wherein the sensor includes an electromagnetic sensor.
 25. The systemof claim 21 wherein the sensor includes a fiber optic shape sensor. 26.The system of claim 21 wherein the processor is further configured fordetermining a relationship between a first sensor record of the set ofordered sensor records, a second sensor record of the set of orderedsensor records, a first candidate match point defined within the modelof the anatomical passageway, and a second candidate match point definedwithin the model of the anatomical passageway.
 27. The system of claim26 wherein determining the relationship includes determining atransition probability between the first and second candidate matchpoints.
 28. The system of claim 21 wherein the processor is furtherconfigured for tracking the movement of the medical instrument relativeto the model of the anatomical passageway formation.
 29. The system ofclaim 21 wherein the set of ordered sensor records include a set oftemporally ordered sensor records.
 30. The system of claim 21 whereinthe set of ordered sensor records include a set of spatially orderedsensor records.
 31. The system of claim 21 wherein receiving the modelof the anatomical passageway formation includes receiving the modelformed from a set of three-dimensional volumetric images.
 32. The systemof claim 21 wherein receiving a set of ordered sensor records includesreceiving the set from an electromagnetic sensor coupled to the medicalinstrument.
 33. The system of claim 21 wherein the set of ordered sensorrecords includes temporally ordered pose observations for a tip of themedical instrument.
 34. The system of claim 21 wherein the set ofordered sensor records includes temporally ordered shape observationsfor a portion of the medical instrument.
 35. The system of claim 21wherein registering includes filtering the received set of orderedsensor records.
 36. The system of claim 35 wherein filtering thereceived set of ordered sensor records includes creating a subset of theordered sensor records that includes only sensor data recorded at aselected phase of anatomical movement.
 37. The system of claim 36wherein the selected phase of anatomical movement is an expiration phaseof a breathing cycle of a lung.
 38. The system of claim 27 whereindetermining the transition probability includes determining a distancebetween the first and second candidate match points.
 39. The system ofclaim 27 wherein determining the transition probability includesdetermining a distance between a first sensor record associated with thefirst candidate match point and a second sensor record associated withthe second candidate match point.
 40. The system of claim 27 whereindetermining the transition probability includes determining whether thefirst and second candidate match points are in contiguous passageways ofthe model of the anatomical passageway formation.
 41. The system ofclaim 21 wherein the processor is further configured for spatiallysubsampling the set of ordered sensor records.
 42. The system of claim21 wherein registering includes comparing position data from at leastone of the sensor records in the set of ordered sensor records to atleast one of the candidate match points defined within a passageway inthe model of the anatomical passageway formation.
 43. The system ofclaim 21 wherein registering includes comparing orientation vector datafrom at least one of the sensor records in the set of ordered sensorrecords to at least one of the candidate match points defined within apassageway in the model of the anatomical passageway formation.
 44. Thesystem of claim 21 further comprising a display for displaying aninternal virtual view of the anatomical passageway formation.